Quality control for images and video is often implemented based on subjective testing using people that compare different images or video content. A quality estimator can be used to compute a score to estimate perceived quality of a single input image. When comparing image quality estimate scores of two images, there can be is uncertainty as to how to compare such scores or what fraction of viewers might actually prefer the image with the lower score.